PS 





MAIL STOP AF 
EXPEDITED PROCEDURE 
GROUP ART UNIT 2672 



^er /£ / 



02355.012108 



PATENT APPLICATION 



IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



In re Application of: 
KIYOHIDE SATOH, ET AL. 
Appln.No.: 09/658,463 
Filed: September 8, 2000 



For: AUGMENTED REALITY ) 
PRESENTATION APPARATUS AND : 
METHOD, AND STORAGE MEDIUM ) 



Commissioner for Patents 
MAIL STOP AF 
P.O. Box 1450 
Alexandria, VA 22313-1450 



Examiner: J. Wang 
Group Art Unit: 2672 

RECEIVED 

MAR 1 7 2004 

March i s, 2004 Technology Center 2600 



REQUEST FOR RECONSIDERATION 



Sir: 

In response to the Official Action mailed January 1 5, 2004, Applicants 
respectfully request reconsideration and allowance in view of the following remarks. 

Claims 1, 3, 4, 6-10, 12-19, 24-29, 31 and 32 remain pending in the 
application, with Claims 1, 10, 19, 31 and 32 being independent. 




Claims 1, 3, 4, 6-10, 12-19, 24-29, 31 and 32 were rejected under 35 U.S.C. 
§ 103 as being unpatentable over U.S. Patent No. 6,522,312 ( Ohshima et al. ) in view of 
U.S. Patent No. 6,624,853 ( Latvpov ) and U.S. Patent No. 6,445,815 ( Sato ). This rejection 
is respectfully traversed. 

As is recited in independent Claim 1, the present invention relates to an 
augmented reality presentation apparatus for superimposing a virtual object in a real space, 
characterized by including objective viewpoint augmented reality presentation means for 
superimposing the virtual object viewed from a first viewpoint position, which differs from 
any player's viewpoint position, in the real space viewed from the first viewpoint position. 
The objective viewpoint augmented reality presentation means includes first video sensing 
means, first video generation means, first video composition means and objective 
viewpoint video display means. The first video sensing means senses a video of the real 
space viewed from the first viewpoint position. The first video generation means generates 
a video of the virtual object viewed from the first viewpoint position. The first video 
composition means composes an augmented reality video viewed from the first viewpoint 
position on the basis of the videos of the real space and the virtual object viewed from the 
first viewpoint position. The objective viewpoint video display means displays the 
augmented reality video obtained from the first video composition means on a screen of a 
predetermined display apparatus. The apparatus further includes player's viewpoint 
augmented reality presentation means for superimposing the virtual object viewed from the 
player's viewpoint position in the real space viewed from the player's viewpoint position. 
The player's viewpoint augmented reality presentation means includes second video 
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sensing means, player's viewpoint position acquiring means, second video generation 
means, second video composition means and display means. The second video sensing 
means senses a video of the real space viewed from the player's viewpoint position. The 
player's viewpoint position acquiring means acquires information indicating the player's 
viewpoint position. The second video generation means generates a video of the virtual 
object viewed from the player's viewpoint position using the information indicating the 
player's viewpoint position. The second video composition means composes an 
augmented reality video viewed from the player's viewpoint position on the basis of the 
videos of the real space and the virtual object viewed from the player's viewpoint position. 
The display means displays to the player the augmented reality video viewed from the 
player's viewpoint position on a screen of a player's display apparatus separate from the 
predetermined display apparatus. 

With the above arrangement, an augmented reality video can be generated 
according to a player's viewpoint and displayed on a screen in a player's head mounted 
display and an augmented reality video can be generated according to another viewpoint 
different from the player's viewpoint and displayed on a screen of a predetermined display 
apparatus different from the player's head mounted display. This augmented reality video 
according to the other viewpoint can be viewed by a third party other than the player. 

Ohshima et al. is directed to an apparatus for presenting mixed reality 
shared among operators. As shown in Fig. 2, an air hockey game can be presented in 
mixed reality space for two players. The table 1000 and mallets 260L, 260R are real 
objects and the puck is a virtual object. As understood by Applicants, TV camera 230 is 
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fixed above the center of table 1000 and can capture the entire surface of the table 1000 
within its field of view. Mallet position measurement unit 5010 detects the position of the 
mallets from an image captured by TV camera 230 and outputs a result of the detection to 
game status management unit 5030. This unit determines the next position of the puck 
based on the detected mallet positions and a current managed position of the puck. The 
computer generation of the puck according to position and posture of each player is 
generated, and is outputted to the head mounted display 210 after being superimposed on 
the image captured by camera 230. 

That is, as understood by Applicants, in Ohshima et al. the image captured 
by TV camera 230 is not an image to be presented to a third party other than a player, but is 
used for determining the next position of the mallet 260 by the mallet position measuring 
unit 5010. Ohshima et al. may disclose a technique for generating an image of a player's 
viewpoint and displaying the generated image to the player's head mounted display, but 
Ohshima et al fails to disclose or suggest superimposing a virtual object viewed from a 
first viewpoint position, which differs from any player's viewpoint position, in a real space 
viewed from the first viewpoint position, and displaying an augmented reality video 
viewed from the first viewpoint position on a screen of a predetermined display apparatus, 
as is recited in independent Claim 1 . 

Thus, Ohshima et al. fails to disclose or suggest important features of the 
present invention recited in this independent claim. 

Latypov relates to a method and system for creating video programs in a 
virtual studio. An image according to a position and orientation of an actor can be 




displayed. Such image is obtained by superimposing a virtual image with an image 
captured by a TV camera. However, like Ohshima et aL Latvpov merely discloses 
generating an image according to the actor's position and orientation (corresponding to the 
player's viewpoint) and displaying the image generated based upon the position and 
orientation of the actor. Latypov does not teach generating an image according to a 
viewpoint other than a player's viewpoint and displaying the image on a predetermined 
display apparatus separate from a player's display apparatus. Therefore, Latvpov fails to 
remedy the deficiencies of Ohshima et al. noted above with respect to the independent 
claims. 

Sato has also been reviewed but is not believed to be any more relevant than 
the citations discussed above. 

For the foregoing reasons, Applicants submit that Claim 1 is patentable over 
the citations of record. Independent Claims 10, 19, 31 and 32, recite similar features, and 
are believed to be patentable for similar reasons. Reconsideration and withdrawal of the 
§ 103 rejection are respectfully requested. 

For the foregoing reasons, Applicants respectfully submit that the present 
invention is patentably defined by independent Claims 1, 10, 19, 31 and 32. Dependent 
Claims 3, 4, 6-9, 12-18 and 24-29 are also allowable, in their own right, for defining 
features of the present invention in addition to those recited in their respective independent 
claims. Individual consideration of the dependent claims is requested. 
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Applicants submit that the present application is in condition for allowance. 
Favorable reconsideration, withdrawal of the rejection set forth in the above-noted Office 
Action, and an early Notice of Allowance are requested. 

Applicants' undersigned attorney may be reached in our Washington, D.C. 
office by telephone at (202) 530-1010. All correspondence should continue to be directed 
to our below-listed address. 



Respectfully submitted, 




Attorney for Applicants 
Registration No. 33,628 



FITZPATRICK, CELLA, HARPER & SCINTO 
30 Rockefeller Plaza 
New York, New York 101 12-3801 
Facsimile: (212)218-2200 

MAW\tnt 
DC_MAIN 160490V1 
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